Machine Learning for Robot Grasping and Manipulation

Robotics as a technology has an incredible potential for improving our everyday lives. Robots could perform household chores, such as cleaning, cooking, and gardening, in order to give us more time for other pursuits. Robots could also be used to perform tasks in hazardous environments, such as turning off a valve in an emergency or safely sorting our more dangerous trash. However, all of these applications would require the robot to perform manipulation tasks with various objects. Today's robots are used primarily for performing specialized tasks in controlled scenarios, such as manufacturing. The robots that are used in today's applications are typically designed for a single purpose and they have been preprogrammed with all of the necessary task information. In contrast, a robot working in a more general environment will often be confronted with new objects and scenarios. Therefore, in order to reach their full potential as autonomous physical agents, robots must be capable of learning versatile manipulation skills for different objects and situations. Hence, we have worked on a variety of manipulation skills to improve those capabilities of robots, and the results have lead to several new approaches, which are presented in this thesis Learning manipulation skills is, however, an open problem with many challenges that still need to be overcome. The first challenge is to acquire and improve manipulation skills with little to no human supervision. Rather than being preprogrammed, the robot should be able to learn from human demonstrations and through physical interactions with objects. Learning to improve skills through trial and error learning is a particularly important ability for an autonomous robot, as it allows the robot to handle new situations. This ability also removes the burden from the human demonstrator to teach a skill perfectly, as a robot is allowed to make mistakes if it can learn from them. In order to address this challenge, we present a continuum-armed bandits approach for learning to grasp objects. The robot learns to predict the performances of different grasps, as well as how certain it is of this prediction, and selects grasps accordingly. As the robot tries more grasps, its predictions become more accurate, and its grasps improve accordingly. A robot can master a manipulation skill by learning from different objects in various scenarios. Another fundamental challenge is therefore to efficiently generalize manipulations between different scenarios. Rather than relearning from scratch, the robot should find similarities between the current situation and previous scenarios in order to reuse manipulation skills and task information. For example, the robot can learn to adapt manipulation skills to new objects by finding similarities between them and known objects. However, only some similarities between objects will be relevant for a given manipulation. The robot must therefore also learn which similarities are important for adapting the manipulation skill. We present two object representations for generalizing between different situations. Contacts between objects are important for many manipulations, but it is difficult to define general features for representing sets of contacts. Instead, we define a kernel function for comparing contact distributions, which allows the robot to use kernel methods for learning manipulations. The second approach is to use warped parameters to define more abstract features, such as areas and volumes. These features are defined as functions of known object models. The robot can compute these parameters for novel objects by warping the shape of the known object to match the unknown object. Learning about objects also requires the robot to reconcile information from multiple sensor modalities, including touch, hearing, and vision. While some object properties will only be observed by specific sensor modalities, other object properties can be determined from multiple sensor modalities. For example, while color can only be determined by vision, the shape of an object can be observed using vision or touch. The robot should use information from all of its senses in order to quickly learn about objects. We explain how the robot can learn low-dimensional representations of tactile data by incorporating cues from vision data. As touching an object usually occludes the surface, the proposed method was designed to work with weak pairings between the data in the two sensor modalities. The robot can also learn more efficiently if it reuses skills between different tasks. Rather than relearn a skill for each new task, the robot should learn manipulation skills that can be reused for multiple tasks. For an autonomous robot, this would require the robot to divide tasks into smaller steps. Dividing tasks into smaller parts makes it easier to learn the corresponding skills. If a step is a part of many tasks, then the robot will have more opportunities to practice the associated skill, and more tasks will benefit from the resulting performance improvement. In order to learn a set of useful subtasks, we propose a probabilistic model for dividing manipulations into phases. This model captures the conditions for transitioning between different phases, which represent subgoals and constraints of the overall tasks. The robot can use the model together with model-based reinforcement learning in order to learn skills for moving between phases. When confronted with a new task, the robot will have to select a suitable sequence of skills to execute. The robot must therefore also learn to select which manipulation to execute in the current scenario. Selecting sequences of motor primitives is difficult, as the robot must take into consideration the current task, state, and future actions when selecting the next motor skill to execute. We therefore present a value function method for selecting skills in an optimal manner. The robot learns the value function for the continuous state space using a flexible non-parametric model-based approach. Learning manipulation skills also poses certain challenges for learning methods. The robot will not have thousands of samples when learning a new manipulation skill, and must instead actively collect new samples or use data from similar scenarios. The learning methods presented in this thesis are, therefore, designed to work with relatively small amounts of data, and can generally be used during the learning process. Manipulation tasks also present a spectrum of different problem types. Hence, we present supervised, unsupervised, and reinforcement learning approaches in order to address the diverse challenges of learning manipulations skills

Bim and Bmf synergize to induce apoptosis in Neisseria gonorrhoeae infection

Abstract: Bcl-2 family proteins including the pro-apoptotic BH3-only proteins are central regulators of apoptotic cell death. Here we show by a focused siRNA miniscreen that the synergistic action of the BH3-only proteins Bim and Bmf is required for apoptosis induced by infection with Neisseria gonorrhoeae (Ngo). While Bim and Bmf were associated with the cytoskeleton of healthy cells, they both were released upon Ngo infection. Loss of Bim and Bmf from the cytoskeleton fraction required the activation of Jun-N-terminal kinase-1 (JNK-1), which in turn depended on Rac-1. Depletion and inhibition of Rac-1, JNK-1, Bim, or Bmf prevented the activation of Bak and Bax and the subsequent activation of caspases. Apoptosis could be reconstituted in Bim-depleted and Bmf-depleted cells by additional silencing of antiapoptotic Mcl-1 and Bcl-XL, respectively. Our data indicate a synergistic role for both cytoskeletal-associated BH3-only proteins, Bim, and Bmf, in an apoptotic pathway leading to the clearance of Ngo-infected cells. Author Summary: A variety of physiological death signals, as well as pathological insults, trigger apoptosis, a genetically programmed form of cell death. Pathogens often induce host cell apoptosis to establish a successful infection. Neisseria gonorrhoeae (Ngo), the etiological agent of the sexually transmitted disease gonorrhoea, is a highly adapted obligate human-specific pathogen and has been shown to induce apoptosis in infected cells. Here we unveil the molecular mechanisms leading to apoptosis of infected cells. We show that Ngo-mediated apoptosis requires a special subset of proapoptotic proteins from the group of BH3-only proteins. BH3-only proteins act as stress sensors to translate toxic environmental signals to the initiation of apoptosis. In a siRNA-based miniscreen, we found Bim and Bmf, BH3-only proteins associated with the cytoskeleton, necessary to induce host cell apoptosis upon infection. Bim and Bmf inactivated different inhibitors of apoptosis and thereby induced cell death in response to infection. Our data unveil a novel pathway of infection-induced apoptosis that enhances our understanding of the mechanism by which BH3-only proteins control apoptotic cell death

Bacterial porin disrupts mitochondrial membrane potential and sensitizes host cells to apoptosis

The bacterial PorB porin, an ATP-binding beta-barrel protein of pathogenic Neisseria gonorrhoeae, triggers host cell apoptosis by an unknown mechanism. PorB is targeted to and imported by host cell mitochondria, causing the breakdown of the mitochondrial membrane potential (delta psi m). Here, we show that PorB induces the condensation of the mitochondrial matrix and the loss of cristae structures, sensitizing cells to the induction of apoptosis via signaling pathways activated by BH3-only proteins. PorB is imported into mitochondria through the general translocase TOM but, unexpectedly, is not recognized by the SAM sorting machinery, usually required for the assembly of beta-barrel proteins in the mitochondrial outer membrane. PorB integrates into the mitochondrial inner membrane, leading to the breakdown of delta psi m. The PorB channel is regulated by nucleotides and an isogenic PorB mutant defective in ATP-binding failed to induce delta psi m loss and apoptosis, demonstrating that dissipation of delta psi m is a requirement for cell death caused by neisserial infection

Measurement of the Bottom-Strange Meson Mixing Phase in the Full CDF Data Set

We report a measurement of the bottom-strange meson mixing phase \beta_s using the time evolution of B0_s -> J/\psi (->\mu+\mu-) \phi (-> K+ K-) decays in which the quark-flavor content of the bottom-strange meson is identified at production. This measurement uses the full data set of proton-antiproton collisions at sqrt(s)= 1.96 TeV collected by the Collider Detector experiment at the Fermilab Tevatron, corresponding to 9.6 fb-1 of integrated luminosity. We report confidence regions in the two-dimensional space of \beta_s and the B0_s decay-width difference \Delta\Gamma_s, and measure \beta_s in [-\pi/2, -1.51] U [-0.06, 0.30] U [1.26, \pi/2] at the 68% confidence level, in agreement with the standard model expectation. Assuming the standard model value of \beta_s, we also determine \Delta\Gamma_s = 0.068 +- 0.026 (stat) +- 0.009 (syst) ps-1 and the mean B0_s lifetime, \tau_s = 1.528 +- 0.019 (stat) +- 0.009 (syst) ps, which are consistent and competitive with determinations by other experiments.Comment: 8 pages, 2 figures, Phys. Rev. Lett 109, 171802 (2012

The challenge to verify ceramide's role of apoptosis induction in human cardiomyocytes - a pilot study

<p>Abstract</p> <p>Background</p> <p>Cardioplegia and reperfusion of the myocardium may be associated with cardiomyocyte apoptosis and subsequent myocardial injury. In order to establish a pharmacological strategy for the prevention of these events, this study aimed to verify the reliability of our human cardiac model and to evaluate the pro-apoptotic properties of the sphingolipid second messenger ceramide and the anti-apoptotic properties of the acid sphingomyelinase inhibitor amitryptiline during simulated cardioplegia and reperfusion ex vivo.</p> <p>Methods</p> <p>Cardiac biopsies were retrieved from the right auricle of patients undergoing elective CABG before induction of cardiopulmonary bypass. Biopsies were exposed to <it>ex vivo </it>conditions of varying periods of cp/rep (30/10, 60/20, 120/40 min). Groups: I (untreated control, n = 10), II (treated control cp/rep, n = 10), III (cp/rep + ceramide, n = 10), IV (cp/rep + amitryptiline, n = 10) and V (cp/rep + ceramide + amitryptiline, n = 10). For detection of apoptosis anti-activated-caspase-3 and PARP-1 cleavage immunostaining were employed.</p> <p>Results</p> <p>In group I the percentage of apoptotic cardiomyocytes was significantly (p < 0.05) low if compared to group II revealing a time-dependent increase. In group III ceramid increased and in group IV amitryptiline inhibited apoptosis significantly (p < 0.05). In contrast in group V, under the influence of ceramide and amitryptiline the induction of apoptosis was partially suppressed.</p> <p>Conclusion</p> <p>Ceramid induces and amitryptiline suppresses apoptosis significantly in our ex vivo setting. This finding warrants further studies aiming to evaluate potential beneficial effects of selective inhibition of apoptosis inducing mediators on the suppression of ischemia/reperfusion injury in clinical settings.</p

Metabolic regulation by p53

We are increasingly aware that cellular metabolism plays a vital role in diseases such as cancer, and that p53 is an important regulator of metabolic pathways. By transcriptional activation and other means, p53 is able to contribute to the regulation of glycolysis, oxidative phosphorylation, glutaminolysis, insulin sensitivity, nucleotide biosynthesis, mitochondrial integrity, fatty acid oxidation, antioxidant response, autophagy and mTOR signalling. The ability to positively and negatively regulate many of these pathways, combined with feedback signalling from these pathways to p53, demonstrates the reciprocal and flexible nature of the regulation, facilitating a diverse range of responses to metabolic stress. Intriguingly, metabolic stress triggers primarily an adaptive (rather than pro-apoptotic) p53 response, and p53 is emerging as an important regulator of metabolic homeostasis. A better understanding of how p53 coordinates metabolic adaptation will facilitate the identification of novel therapeutic targets and will also illuminate the wider role of p53 in human biology

The flavonoid 4,4′-dimethoxychalcone promotes autophagy-dependent longevity across species

Ageing constitutes the most important risk factor for all major chronic ailments, including malignant, cardiovascular and neurodegenerative diseases. However, behavioural and pharmacological interventions with feasible potential to promote health upon ageing remain rare. Here we report the identification of the flavonoid 4,4′- dimethoxychalcone (DMC) as a natural compound with anti-ageing properties. External DMC administration extends the lifespan of yeast, worms and flies, decelerates senescence of human cell cultures, and protects mice from prolonged myocardial ischaemia. Concomitantly, DMC induces autophagy, which is essential for its cytoprotective effects from yeast to mice. This pro-autophagic response induces a conserved systemic change in metabolism, operates independently of TORC1 signalling and depends on specific GATA transcription factors. Notably, we identify DMC in the plant Angelica keiskei koidzumi, to which longevity- and health-promoting effects are ascribed in Asian traditional medicine. In summary, we have identified and mechanistically characterised the conserved longevity-promoting effects of a natural anti-ageing drug

Molecular mechanisms of cell death: recommendations of the Nomenclature Committee on Cell Death 2018.

Over the past decade, the Nomenclature Committee on Cell Death (NCCD) has formulated guidelines for the definition and interpretation of cell death from morphological, biochemical, and functional perspectives. Since the field continues to expand and novel mechanisms that orchestrate multiple cell death pathways are unveiled, we propose an updated classification of cell death subroutines focusing on mechanistic and essential (as opposed to correlative and dispensable) aspects of the process. As we provide molecularly oriented definitions of terms including intrinsic apoptosis, extrinsic apoptosis, mitochondrial permeability transition (MPT)-driven necrosis, necroptosis, ferroptosis, pyroptosis, parthanatos, entotic cell death, NETotic cell death, lysosome-dependent cell death, autophagy-dependent cell death, immunogenic cell death, cellular senescence, and mitotic catastrophe, we discuss the utility of neologisms that refer to highly specialized instances of these processes. The mission of the NCCD is to provide a widely accepted nomenclature on cell death in support of the continued development of the field

Guidelines for the use and interpretation of assays for monitoring autophagy (3rd edition)

In 2008 we published the first set of guidelines for standardizing research in autophagy. Since then, research on this topic has continued to accelerate, and many new scientists have entered the field. Our knowledge base and relevant new technologies have also been expanding. Accordingly, it is important to update these guidelines for monitoring autophagy in different organisms. Various reviews have described the range of assays that have been used for this purpose. Nevertheless, there continues to be confusion regarding acceptable methods to measure autophagy, especially in multicellular eukaryotes. For example, a key point that needs to be emphasized is that there is a difference between measurements that monitor the numbers or volume of autophagic elements (e.g., autophagosomes or autolysosomes) at any stage of the autophagic process versus those that measure fl ux through the autophagy pathway (i.e., the complete process including the amount and rate of cargo sequestered and degraded). In particular, a block in macroautophagy that results in autophagosome accumulation must be differentiated from stimuli that increase autophagic activity, defi ned as increased autophagy induction coupled with increased delivery to, and degradation within, lysosomes (inmost higher eukaryotes and some protists such as Dictyostelium ) or the vacuole (in plants and fungi). In other words, it is especially important that investigators new to the fi eld understand that the appearance of more autophagosomes does not necessarily equate with more autophagy. In fact, in many cases, autophagosomes accumulate because of a block in trafficking to lysosomes without a concomitant change in autophagosome biogenesis, whereas an increase in autolysosomes may reflect a reduction in degradative activity. It is worth emphasizing here that lysosomal digestion is a stage of autophagy and evaluating its competence is a crucial part of the evaluation of autophagic flux, or complete autophagy. Here, we present a set of guidelines for the selection and interpretation of methods for use by investigators who aim to examine macroautophagy and related processes, as well as for reviewers who need to provide realistic and reasonable critiques of papers that are focused on these processes. These guidelines are not meant to be a formulaic set of rules, because the appropriate assays depend in part on the question being asked and the system being used. In addition, we emphasize that no individual assay is guaranteed to be the most appropriate one in every situation, and we strongly recommend the use of multiple assays to monitor autophagy. Along these lines, because of the potential for pleiotropic effects due to blocking autophagy through genetic manipulation it is imperative to delete or knock down more than one autophagy-related gene. In addition, some individual Atg proteins, or groups of proteins, are involved in other cellular pathways so not all Atg proteins can be used as a specific marker for an autophagic process. In these guidelines, we consider these various methods of assessing autophagy and what information can, or cannot, be obtained from them. Finally, by discussing the merits and limits of particular autophagy assays, we hope to encourage technical innovation in the field